Multimodal Integration - A Statistical View

Achieving Multimodal Cohesion during Intercultural Conversations

How do English as a lingua franca (ELF) speakers achieve multimodal cohesion on the basis of their specific interests and cultural backgrounds? From a dialogic and collaborative view of communication, this study focuses on how verbal and nonverbal modes cohere together during intercultural conversations. The data include approximately 160-minute transcribed video recordings of ELF interactions ...

A Bayesian view on multimodal cue integration

Multimodal Integration - A Statistical

|This paper presents a statistical approach to developing multimodal recognition systems and, in particular, to integrating the posterior probabilities of parallel input signals involved in the multimodal system. We rst identify the primary factors that innuence multimodal recognition performance by evaluating the multimodal recognition probabilities. We then develop two techniques, an estimate...

Human Factors and Design Issues in Multimodal (Speech/Gesture) Interface

Multimodal interfaces are the emerging technology that offers expressive, transparent, efficient, robust, and mobile human-computer interaction. In this paper, we described the speech/gesture based multimodal interface systematically from the human factors point of view. To design more practical and efficient multimodal interface, human factors issues such as user modeling, usability studies, s...

Multimodal Integration A Biological View

We present a novel methodology for building highly integrated multimodal systems. Our approach is motivated by current cognitive and behavioral theories of sensory perception in animals and humans. We argue that perceptual integration in multimodal systems needs to happen at the lowest levels of the individual perceptual processes. Rather than treating each modality as a separately processed, i...